Within-level Group Factorial Invariance with Multilevel Data: Multilevel Factor Mixture and Multilevel MIMIC Models

This study suggests two approaches to factorial invariance testing with multilevel data when the groups are at the within level: multilevel factor mixture model for known classes (ML FMM) and multilevel multiple indicators multiple causes model (ML MIMIC). The adequacy of the proposed approaches was investigated using Monte Carlo simulations. Additionally, the performance of different types of model selection criteria for determining factorial invariance or in detecting item noninvariance was examined. Generally, both ML FMM and ML MIMIC demonstrated acceptable performance with high true positive and low false positive rates, but the performance depended on the fit statistics used for model selection under different simulation conditions. Within-level Group Factorial Invariance 3 Within-level Group Factorial Invariance in Multilevel Data: Multilevel Factor Mixture Model and Multilevel MIMIC Model For the last several decades, the area of factorial invariance (or measurement invariance in broader contexts) has received great attention. Not only has extensive methodological work been conducted on this topic but testing factorial invariance has become common practice before comparing latent means in applied research (Raykov, Marcoulides, & Li, 2012). However, there are still unresolved issues in this area: for example, locating a truly invariant variable for a reference variable (French & Finch, 2008), establishing partial invariance (e.g., Millsap & Kwok, 2004), and developing practical criteria in determining a lack of factorial invariance, especially with a mean structure (e.g., Fan & Sivo, 2009). With advances in methodology, issues involved in certain analytic methods arise in addition to the previously mentioned general concerns related to factorial invariance testing. For example, special model specification issues occur in factorial invariance testing with multilevel data. This study is particularly interested in model specification issues related to testing factorial invariance for within-level groups in multilevel modeling. It is well known among social scientists that a single level statistical approach to multilevel data underestimates standard errors in statistical significance testing, which may lead to incorrect statistical inference, that is, Type I error. Recently, Kim, Kwok, and Yoon (2012) studied both within-level and between-level factorial invariance testing. In evaluating weak factorial invariance across between-level groups, they conducted multilevel confirmatory factor analysis for multiple groups (i.e., multilevel multigroup CFA; Muthén, 1989). Their simulation results supported the suitability of multilevel multigroup CFA with acceptable power and adequate Type I error control whereas the single-level multigroup CFA yielded inflated Type I Within-level Group Factorial Invariance 4 error rates as a function of the intraclass correlation (ICC) and cluster size. Thus, they recommended multilevel multigroup CFA for factorial invariance testing with multilevel data. In conducting within-level factorial invariance testing, however, Kim et al. suggested a designbased approach (Muthén & Satorra, 1995; Wu & Kwok, 2012) using the TYPE = COMPLEX option in Mplus (Muthén & Muthén, 2012) because constructing multigroup multilevel models for within-level groups is not feasible when the group indicator (e.g., females and males within schools) is crossed across higher-level clusters. For each within-level group they generated a single factor model at both the withinand between-levels with an identical set of factor loadings for both levels. When noninvariance was simulated in one of the factor loadings in the withinlevel groups, the design-based approach perfectly detected the violation of weak factorial invariance (power = 1.0). Under complete invariance, Type I error rates were around the nominal level (.04 ~ .07). The design-based approach to multilevel data is in fact a single level CFA, but corrects the underestimated standard errors of parameter estimates due to data dependency. Of note is that the design-based approach implicitly assumes factorial invariance across levels (i.e., cross-level factorial invariance) by constructing a single-level CFA model. Cross-level factorial invariance 1 is referred to as the equivalence of factor loadings with the same number of factors across the between and within levels (Dedrick & Greenbaum, 2011). However, cross-level factorial invariance is not always warranted for the application of the design-based approach. Wu and Kwok (2012) studied the performance of the design-based approach in a single group context when the between-level factor structure was different from the within-level factor structure. When cross-level factorial invariance was violated especially with a simple within factor structure (e.g., a one-factor model) and a complex between factor structure (e.g., a two-factor Within-level Group Factorial Invariance 5 model), the design-based approach showed poor model fit to the data and yielded biased estimates of both fixed and random effects. Given the stringent assumptions of the design-based approach, the purpose of this study is to propose two potential approaches to factorial invariance testing with multilevel data when the groups are at the within level: multilevel factor mixture model for known classes (ML FMM) and multilevel multiple indicators multiple causes model (ML MIMIC). To this end, we conducted three Monte Carlo studies. First, we investigated the adequacy of ML FMM in testing weak and strong factorial invariance across within-level groups at the scale level. Next, the performance of ML MIMIC in detecting a noninvariant item at the item level was examined. Note that ML FMM and ML MIMIC were used for slightly different purposes: the first to establish weak or strong factorial invariance, the latter to detect a particular noninvariant variable. We do not purport to compare the two methods but rather we explore their performance under the circumstances in which they are typically employed for factorial invariance testing. In the third study, the proposed multilevel approach to factorial invariance testing, specifically, ML FMM was compared to the single level design-based approach when between and within factor structures were not identical. Throughout these Monte Carlo studies, different types of model selection criteria were examined with respect to their performance in determining a level of factorial invariance (e.g., weak invariance) or in detecting a noninvariant item. Finally, the proposed methods of factorial invariance testing across within-level groups were illustrated with the mathematics self-efficacy measure from the Programme for International Student Assessment (PISA) 2003 data (OECD, 2005). Multilevel factor mixture model for known classes Within-level Group Factorial Invariance 6 Factor mixture modeling is used to analyze unobserved population heterogeneity and identify latent classes underlying the observed data. In addition, factor mixture models can be used for observed classes, which are, in effect, analogous to multiple group analysis because all latent classes identified in the model correspond to the observed groups. However, multiple group analysis under the factor mixture framework expands modeling capabilities beyond the conventional SEM. Some of the modeling issues as discussed in the previous section (modeling factorial invariance across within-level groups in multilevel data) can be solved by incorporating categorical latent variables into the analysis. In factor mixture models, the observed random variable y is modeled conditional on the latent class variable, C (C = 1, 2, ..., c) (Asparouhov & Muthén, 2008; Lubke & Muthén, 2005): [ | ] [ | ] In Equation 1, Λ is the factor loading matrix expressing the relations of the observed outcome variables y with the latent variables η, Γy is the pattern coefficients of y regressed on observed covariates x, and ν and ε are intercepts and residuals, respectively. Equation 2 shows the relations of the endogenous latent variables η to exogenous latent variables η (Β) and to observed covariates x (Γη) with μ and ζ as intercepts and residuals, correspondingly. Conditional on the latent classes, y is assumed to be multivariate normally distributed. Because the latent classes are unordered categories, the latent class membership can be defined as a multinomial variable (or a binary variable for two classes) by